Enhancing a Rendering System to Distinguish Presentation Time from Data Time

ABSTRACT

A machine-implemented method: (A) maintains a value of a data time parameter representing an amount of time required by a rendering system to render a portion of temporal sequence presentation data at a default presentation rate; (B) provides the value of the data time parameter to a first component of the rendering system; (C) calculates, based on the value of the data time parameter, a value of a presentation time parameter representing an amount of time elapsed during rendering of the portion of the temporal sequence presentation data; and (D) provides the value of the presentation time parameter to a second component of the rendering system; wherein the value of the presentation time parameter is not equal to the value of the data time parameter.

BACKGROUND Field of the Invention

The present invention relates to the field of content presentation by adigital rendering system such as a digital media player.

Related Art

Most traditional digital rendering systems, such as RealNetworks®RealPlayer® digital media players, maintain an internal variable duringplayback of media content that reflects a current presentation time(hereafter referred to as “Current Time”). Current Time is, in effect, acurrent “position” in the media content that is being displayed andrendered. Typically Current Time is set to zero at the beginning of themedia content, and it reaches a measure of time equal to a duration ofpresentation of the content of the entire work when the end of the mediacontent is reached.

In most traditional players, such as the RealPlayer® digital mediaplayer, a Current Time value is: (a) regularly calculated by a singlemodule; (b) acquired and stored by core routines of the player; and (c)distributed to, and utilized by, various internal player objects. Theseinternal objects utilize the Current Time value to determine when it istime to initiate or terminate various tasks associated with mediacontent playback. The calculation of a Current Time value by the singlemodule, and the distribution to, and utilization by, multiple objectswithin a player of the same Current Time value has a desirable result ofkeeping all objects synchronized.

Typically the Current Time value must be regularly and accuratelyupdated by the player, or the presentation of media content will befaulty. For instance, if the Current Time value is not updatedsufficiently often, a video component of a media stream may appearuneven or jumpy, and gaps in an audio content may be audible.

Although the concept of Current Time seems straightforward, in fact, itconflates two subtly different properties of media playback. The firstproperty of media playback that is conflated in the concept of CurrentTime is a time elapsed since the beginning of the media contentpresentation (hereafter called “Presentation Time”). Thus, if the mediahas been playing for one minute, the value of Presentation Time is60,000 milliseconds. All of time values discussed herein can be measuredin various units. Two popular units are milliseconds, andcenti-nanoseconds, or 1/10,000,000 of a second. The unit of measurementis not an issue here. Other considerations of representing time that arenot issues here are the precision, the range of values, and the formatof the representation.

The second property of media playback that is conflated in the conceptof Current Time is a location in the media content stream that iscurrently being played (hereafter called “Content Time”). In atraditional linear media stream that is always played back at a fixed,“normal” rate, any given content element is always presented after afixed amount of time has elapsed from the beginning of playback. Becauseof this, each such content element can be regarded as having a timestampassociated with it, i.e., a time value specifying how long it would taketo reach that location, starting from the beginning of the mediacontent, and playing at normal rate. Hereinafter we will call thisproperty “Data Time.”

Presentation Time and Data Time are identical in traditional players,because traditional players can only present media content at a fixed“normal” rate. However, when a player is enhanced with a Time-ScaleModification (TSM) capability, it can present media content at variousrates. Because of this, Presentation Time and Data Time are no longerthe same. For example, if a 60-second clip of media content is presentedat a fixed rate that is twice normal rate, at the end of the clip theData Time is 60,000 milliseconds, but the Presentation Time is 30,000milliseconds. This is because it only takes 30 seconds to play the60-second clip.

We have discovered that problems may occur when a traditional player isenhanced with TSM functionality. In particular, if a Current Time valueis distributed to multiple objects, some of them may interpret theCurrent Time value as specifying Data Time, some of them may interpretthe Current Time value as specifying Presentation Time, and some of themmay interpret the Current Time value as specifying both Data andPresentation Time. Thus, a first problem occurring when a traditionalplayer is enhanced with TSM functionality is that the significance ofthe time value distributed to multiple objects is, in general,ambiguous. A second problem occurring when a traditional player isenhanced with TSM functionality is that Data Time does not, in general,equal Presentation Time, and the calculation, storage, and distributionof a single time value is inadequate to specify both values.

It is quite common for media players to rely on an audio renderer (forexample, a player object responsible for outputting audio contentthrough, for example, a computer sound card) to calculate and update theCurrent Time value. This is done because the nature of audiorepresentation is such that typically each audio data element has eithera fixed, or explicitly specified presentation duration, associated withit, and these presentation durations are enforced by audio renderinghardware. Therefore, the audio renderer can typically determinePresentation Time either by maintaining a running total of thepresentation durations of all audio data elements rendered sinceplayback began, or in some cases by querying the audio renderinghardware itself for the equivalent value.

If a media player does in fact acquire the Current Time value from theaudio renderer, the value that the audio renderer will return to thesystem will typically be the Presentation Time. Since most of the restof the system needs Data Time, most of the rest of the system can nolonger employ the value returned by the audio renderer object.

As one can readily appreciate from the above, a need exists in the artfor a method and apparatus for solving one or more of theabove-described problems.

SUMMARY

Techniques are provided for managing Presentation Time in a digitalrendering system for presentation of temporally-ordered data when thedigital rendering system includes a Variable Rate Presentationcapability. In one embodiment, Presentation Time is converted to DataTime, and Data Time is reported instead of Presentation Time when onlyone time can be reported. In another embodiment, a predetermined one ofPresentation Time and Data Time is returned in response to a request fora Current Time.

In one embodiment of the present invention, a method is provided forrendering temporal sequence presentation data in a rendering system. Themethod includes steps of: (A) receiving a request from a first componentof the rendering system for a first current time; (B) identifying avalue of a presentation time parameter representing an amount of timeelapsed during rendering of a portion of the temporal sequencepresentation data; (C) providing the value of the presentation timeparameter to the first component in response to the request; (D)receiving a request from a second component of the rendering system fora second current time; (E) identifying a value of a data time parameterrepresenting an amount of time required to render the portion of thetemporal sequence presentation data at a default presentation rate; and(F) providing the value of the data time parameter to the secondcomponent in response to the request; (G) wherein the presentation timeparameter value and the data time parameter value represent differenttimes.

In another embodiment of the present invention, a method is provided foruse in a rendering system for rendering temporal sequence presentationdata. The method comprising steps of: (A) initializing a presentationtime parameter value to zero, the presentation time parameterrepresenting an amount of time elapsed during rendering of a portion ofthe temporal sequence presentation data; (B) for each element in theportion of the temporal sequence presentation data, performing steps of:(1) identifying a default rendition period of the element; (2)identifying an actual presentation rate of the element; (3) dividing thedefault rendition period of the element by the actual presentation rateof the element; (4) adding the quotient of step (3) to the presentationtime parameter value; and (C) identifying the sum produced by steps (A)and (B) as the presentation time parameter value.

In another embodiment of the present invention, a method is provided foruse in a rendering system for rendering temporal sequence presentationdata. The method includes steps of: (A) receiving an explicit requestfor a value of a presentation time parameter representing an amount oftime elapsed during rendering of a portion of the temporal sequencepresentation data, wherein the amount of time differs from an amount oftime required to render the portion of the temporal sequencepresentation data at a default presentation rate; and (B) providing thevalue of the presentation time parameter in response to the request.

In another embodiment of the present invention, a method is provided foruse in a rendering system for rendering temporal sequence presentationdata. The method includes steps of: (A) receiving an explicit requestfor the value of a data time parameter representing an amount of timerequired to render a portion of the temporal sequence presentation dataat a default presentation rate, wherein the amount of time differs froman amount of time elapsed during rendering of the portion of thetemporal sequence presentation data; and (B) providing the value of thedata time parameter in response to the request.

In another embodiment of the present invention, a method is provided forenhancing a rendering system which renders temporal sequencepresentation data, the rendering system not including distinct currentpresentation time and current data time parameters. The method includessteps of: (A) adding to the rendering system a monitoring module formonitoring values of a current presentation rate parameter that may varyover time, the current presentation rate parameter representing acurrent rate at which the portion of the temporal sequence presentationdata is being rendered; (B) adding to the rendering system a data timemodule to identify, based on the values of the current presentation rateparameter, a current data time parameter value representing an amount oftime required to render the portion of the temporal sequencepresentation data at a default presentation rate; (C) adding to therendering system a presentation rate modification module to render aportion of the temporal sequence presentation data at any of a pluralityof presentation rates; and (D) adding to the rendering system a currenttime module for returning the value of a predetermined one of thepresentation time parameter and the data time parameter in response to arequest for a current time; wherein the presentation time parameter hasa value that differs from the data time parameter value.

In another embodiment of the present invention, a method is provided forenhancing a rendering system which renders temporal sequencepresentation data, the rendering system not including distinct currentpresentation time and current data time parameters. The method includessteps of: (A) adding to the rendering system a monitoring module formonitoring values of a current presentation rate parameter that may varyover time, the current presentation rate parameter representing acurrent rate at which the portion of the temporal sequence presentationdata is being rendered; (B) adding to the rendering system apresentation time module to identify, based on the values of the currentpresentation rate parameter, a current presentation time parameter valuerepresenting an amount of time required to render the portion of thetemporal sequence presentation data at the current rate; (C) adding tothe rendering system a presentation rate modification module to render aportion of the temporal sequence presentation data at any of a pluralityof presentation rates; and (D) adding to the rendering system a currenttime module for returning the value of a predetermined one of thepresentation time parameter and the data time parameter in response to arequest for a current time; wherein the presentation time parameter hasa value that differs from the data time parameter value.

In another embodiment of the present invention, a method is provided foruse in conjunction with a rendering system which renders temporalsequence presentation data. The method includes steps of: (A) receivinga request for a current time of the temporal sequence presentation dataafter a portion of the temporal sequence presentation data has beenrendered by the rendering system; (B) in response to the request,performing steps of: (1) identifying a data time parameter valuerepresenting an amount of time required to render the portion of thetemporal sequence presentation data at a default presentation rate; and(2) providing the data time parameter value in response to the request.

In another embodiment of the present invention, a method is provided foruse in conjunction with a rendering system which renders temporalsequence presentation data. The method includes steps of: (A) receivinga request for a current time of the temporal sequence presentation dataafter a portion of the temporal sequence presentation data has beenrendered by the rendering system; (B) determining, based on a propertyof the request, whether to return a value of a data time parameter or avalue of a presentation time parameter; and (C) returning, in responseto the request, the value determined in step (B).

In another embodiment of the present invention, a method is provided foruse in conjunction with a rendering system which renders temporalsequence presentation data. The method includes steps of: (A) receivinga request for a current time of the temporal sequence presentation dataafter a portion of the temporal sequence presentation data has beenrendered by the rendering system; (B) identifying an initial currenttime of the temporal sequence presentation data; (C) identifying atleast one presentation rate associated with the temporal sequencepresentation data; (D) modifying the initial current time based on theidentified at least one presentation rate to produce a modified currenttime; and (E) providing the modified current time in response to therequest.

In another embodiment of the present invention, a method is provided foruse in conjunction with a rendering system which renders temporalsequence presentation data including a plurality of data samples. Themethod includes steps of: (A) grouping the plurality of data samplesinto a plurality of buffers; (B) associating with each of the pluralityof buffers a corresponding presentation rate; (C) associating with eachof the plurality of buffers a corresponding initial data time; (D)associating with each of the plurality of buffers a correspondinginitial presentation time; (E) associating with each of the plurality ofbuffers a default rendition period; and (F) associating with each of theplurality of buffers an actual rendition period.

Other features and advantages of various aspects and embodiments of thepresent invention will become apparent from the following descriptionand from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of a Presentation System embodied as aRealNetworks® RealPlayer® application running on a computer;

FIG. 2 shows a block diagram of a generalized Presentation systemaccording to one embodiment of the present invention that includesPresentation System Controller Modules, other Presentation Modules(including a Presentation Rate Modification Module), and a number ofRenderers;

FIG. 3A is a flowchart of a method for identifying values ofpresentation-related parameters according to one embodiment of thepresent invention;

FIG. 3B is a flowchart of a method for identifying a current Data Timebased on a current Presentation Time according to one embodiment of thepresent invention;

FIG. 3C is a flowchart of a method for identifying a current Data Timebased on a current element index according to one embodiment of thepresent invention;

FIG. 3D is a flowchart of a method for identifying a currentPresentation Time based on a current Data Time according to oneembodiment of the present invention;

FIG. 3E is a flowchart of a method for identifying a currentPresentation Time based on a current element index according to oneembodiment of the present invention;

FIG. 3F is a flowchart of a method for identifying an UnmodifiedCumulative Rendition Period for an element and a Data Time at thebeginning of the element according to one embodiment of the presentinvention;

FIG. 3G is a flowchart of a method for identifying a Modified CumulativeRendition Period for an element and the Presentation Time at thebeginning of the element according to one embodiment of the presentinvention;

FIG. 4 is a flow chart of a method for satisfying a request for aCurrent Time according to one embodiment of the present invention; and

FIG. 5 is a flowchart of a method for satisfying a request for a CurrentTime according to another embodiment of the present invention.

DETAILED DESCRIPTION

In accordance with one embodiment of the present invention, PresentationSystem 100 (a more general definition of a Presentation System isprovided below) is embodied as a RealNetworks® RealPlayer® applicationrunning on a computer, for example, under some version of the MicrosoftWindows operating system. As shown in FIG. 1, application module 110sends to, and receives from, Player Core object 120, control and statusmessages such as, for example, Play, Pause, Stop, and so forth. TemporalSequence Presentation Data, also referred to herein as PresentationData, (a more general definition of Temporal Sequence Presentation Datais provided below) is embodied as streaming media content and isdelivered to the RealPlayer® application over the Internet, a local-areanetwork (LAN), or from files stored in the computer that is executingthe RealPlayer® application. For example, in accordance with oneembodiment, the Presentation Data, for example, audio data, is receivedby media content source module(s) 130, and are placed in audio mediadata buffers 140 that are made available to Player Core object 120.

As will be defined in more detail below, each data element of thePresentation Data has a Rendition Type that corresponds to a type ofRenderer (a more general definition of Renderer is provided below) thatcan be used to render the data element. For example, for the embodimentdescribed above, the Rendition Types that can be rendered include butare not limited to audio (encoded in various formats), video, stillimages, text, and scripts. In particular, audio is a Time-DistinguishedRendition Type (a more general definition of Time-Distinguished

Rendition Type is provided below). As a result, for this embodiment,audio is organized within the RealPlayer® application in buffers thatcontain, for example, 100-milliseconds of sample values. Further, everybuffer is timestamped, so these buffers are Timestamped Elements (asmore generally described below, this means that the Data Time of theelement is explicitly represented as part of the element), and the timeassociated with the first sample in every buffer is specified inmilliseconds.

In accordance with this embodiment, the Rendition Period (as moregenerally described below, this is the length of time the renderingprocess should last for the element) of the audio buffers is 100milliseconds. In some ways the individual audio samples play the part ofthe data elements as described below. It would be obvious to someone ofordinary skill in the art how to sometimes regard 100 millisecondbuffers of samples and sometimes the individual samples themselves asaudio elements. In accordance with this embodiment, a sample period ofthe individual audio samples is stored in a header that is part of thesample buffer definition (for example, one 10,000^(th) of a second is atypical sample period).

In accordance with this embodiment, an object called TSMAudioDeviceobject 150 combines functions of the Renderer for audio data(TSMAudioDevice Audio Renderer 160) and a Variable Rate PresentationModule (a more general definition of Renderer is provided below)(TSMAudioDevice VRP Module 170). In accordance with this embodiment, theaudio Renderer part of TSMAudioDevice object 150 (i.e., TSMAudioDeviceAudio Renderer 160) is a Timing Renderer (a more general definition of aTiming Renderer is provided below) for Presentation System 100. Notethat the RealNetworks® RealPlayer® application does not include supportfor variable rate playback. However, Plug-In 180 comprises a productcalled a 2×AV Plug-In is available from Enounce, Incorporated of PaloAlto, Calif. When the 2×AV Plug-In is installed on a computer that hashad the RealPlayer® application previously installed, it “plugs into”the RealPlayer® application, and adds variable rate playback capabilitythereto. The 2×AV Plug-In has its own User Interface, which includes aslider that a user can manipulate to adjust playback rate. In operation,the 2×AV Plug-In communicates with TSMAudioDevice object 150 by sendingmessages through an object called State Information Exchange Server 190(“SIX Server 190”).

Thus, in accordance with this embodiment, TSMAudioDevice object 150accepts messages from SIX Server 190 that specify a desired playback orpresentation rate. Playback or presentation rate values can range from0.3 to 3.0 (a rate of 1.0 is normal; a rate of 0.3 is 30% of the normalrate; and a rate of 3.0 is three times faster than the normal speed).TSMAudioDevice object 150 receives SIXExchange messages from SIX Server190, and stores a requested playback rate value contained in thesemessages as a new value of an internal Current Presentation Rateparameter or property. In addition, as shown in FIG. 1, TSMAudioDeviceobject 150 receives buffers 200 of audio data to be rendered (i.e.,played out through the computer's sound card) from Player Core object120. When TSMAudioDevice object 150 receives buffers 200 of audio datato be rendered, it is processed by TSMAudioDevice VRP Module 170.TSMAudioDevice VRP Module 170 processes buffers 200 through a library ofsignal processing routines, for example, a suitable library of signalprocessing routines called the Time Scale Tailor package is availablefrom Enounce, Incorporated of Palo Alto, Calif. In accordance with thisembodiment, this library carries out digital signal processingprocedures on buffers 200 of audio samples that has the effect ofreducing the number of samples in the buffer (when playing faster thanreal time) or increasing the number of samples in the buffer (whenplaying slower than real time), thereby effectively changing theplayback rate. For example, in accordance with this embodiment,processing the buffer using the library decreases or increases thesamples in a particular way so as to leave the perceptual and linguisticinformation in the buffers unchanged, but to change the duration of thebuffers. Additionally, playback rate parameters, unmodified and modifiedbuffer lengths and Rendering Period values, and other time-relatedvalues are calculated by TSMAudioDevice VRP Module 170, and are storedwith each audio buffer. Then, modified audio data buffers 210 are storedby TSMAudioDevice VRP Module 170 for presentation by TSM AudioDeviceAudio Renderer 160. TSM AudioDevice Audio Renderer 160 comprises AudioRenderer Core Object 165 that submits modified buffers 210 forprocessing to the computer's audio electronics, for example, ComputerSound Card 220. For example, as shown in FIG. 1, Core Object 165comprises an interface known as a WAV driver. This interface is definedby Microsoft, and is supported by the Windows operating system.

On a regular basis during playback or presentation, Player Core object120 calls a method implemented by TSMAudioDevice object 150 calledGetCurrentAudioTime( ). This method returns a Current Time inmilliseconds. Additionally, every time a buffer of audio samples isprocessed (for example, buffer 200), TSMAudioDevice object 150 isresponsible for calling a Player Core object 120 method calledOnTimeSync( ) and passing to the Player Core object 120 method theCurrent Time in milliseconds. Player Core object 120 interprets both ofthese times as Data Times. In this embodiment, Presentation System 100(other than TSMAudioDevice object 150) does not support the concept ofPresentation Times that are different than Data Times. To do this, inaccordance with one embodiment of the present invention, TSMAudioDeviceobject 150 carries out methods described below to convert PresentationTime (for example, as reported by its WAV driver Core object routines)into Data Time (as needed by Player Core object 120).

Before describing the methods to convert Presentation Time into DataTime, we present generalizations of the matters described above inconjunction with FIG. 2. In particular, FIG. 2 shows a block diagram ofa generalized Presentation system that includes: Presentation SystemController Modules 400, Other Presentation Modules 410 (including aPresentation Rate Modification Module), and a number of Renderers, forexample, Audio Renderer 420, Video Renderer 430, and Other Renderers440. Further, as shown in FIG. 2, Temporal Sequence Presentation Data450 is applied as input to Other Presentation Modules 410.

As defined herein, a Presentation System means a system or method that:(a) acquires, interprets, decodes, and manages presentation of TemporalSequence Presentation Data (defined below); (b) selects, instantiates,initializes, controls, and monitors Renderers (defined below); (c)initiates a presentation by determining which presentation data elementsare to be submitted first to which Renderers, effecting such submission,and causing the Renderers to begin processing; (d) maintains a CurrentTime parameter, whose value is regularly updated in amonotonically-increasing fashion during linear presentation (the valuemay be set to zero or any other value when presentation is stopped, or ajump is performed to a non-sequential location)—the Presentation Systemmay maintain and update the value of the Current Time parameter byidentifying a Renderer that can reliably maintain and update itsCumulative Rendition Period, and arrange to acquire the value of thatparameter at regular intervals; (e) distributes its Current Timeparameter to other Presentation Modules as needed; and (f) manages thepresentation process, including by determining which data elementsshould be submitted next to which Renderers, and by comparing itsCurrent Time value to the Data Time of those data elements, therebydetermining when to effect such submission.

A digital media player is a common type of Presentation System, butthere are many other types of Presentation Systems. For example, acontroller that processes a script which causes digitally-controlledmanufacturing equipment to make a printed circuit board is also aPresentation System, as is a controller that processes a script whichcauses a robot to perform a dance.

As defined herein, a Renderer is a system or method having the followingcharacteristics: (a) the Renderer processes Temporal SequencePresentation Data (defined below); (b) the Renderer processes dataelements in an ordered sequence in which “earlier” elements areprocessed before “later” elements (the order may be determined by theorder in which the elements are submitted to the Renderer, or by theData Times (defined below) of the elements, or by using othertechniques); (c) processing a data element takes a finite amount of time(possibly but not typically zero) known as the Rendition Period of thedata element; (d) processing a sequence of data elements takes a finiteamount of time directly related to the sum of the Rendition Periods ofthe individual elements, and, potentially, some other factors (theamount of time required to process (render) a sequence of data elementsis called a Cumulative Rendition Period for those elements); and (e) atleast one instance of a Renderer (often associated with rendering ofaudio data) has a capability of reporting back to a module, for example,a Presentation System Control Module, upon request, a current value ofthe Cumulative Rendition Period (a Renderer that is consistently used bythe Presentation System in this fashion is referred to as a TimingRenderer).

As defined herein, Temporal Sequence Presentation Data, also referred toas Presentation Data, means data having the following characteristics:(a) the purpose, utility, or semantics of the data is closely associatedwith its presentation—presentation involves rendering of the data toachieve some effect (including but not limited to constituting a visibleand/or audible presentation that can be monitored by a human being); (b)there are a plurality of rendering processes capable of effecting anappropriate presentation of the data; (c) the data comprises a set ofelements; (d) each data element has a Rendition Type that corresponds toa type of Renderer that can be used to render the data element—somecommon Rendition Types are Pulse Code Modulation (PCM) audio, MPEGvideo, and JPEG images; (e) one or more Rendition Types may beTime-Distinguished Rendition Types—Time-Distinguished Rendition Typesare Rendition Types of Temporal Sequence Presentation Data whoseintrinsic characteristics and whose natural rendition process make thempreferred candidates for defining and maintaining a system-wide CurrentTime parameter (note that most audio Rendition Types areTime-Distinguished Rendition Types); (f) associated with each element isa Data Time—the Data Time of some elements may be explicitly representedas part of the element (such elements are called Timestamped Elements),and the Data Time of some elements may be derivable only by performingor simulating an appropriate rendering process on all or part of thePresentation Data (such elements are called Sequential Elements); (g)the elements have a partial ordering, so that when performing renderingoperations on the data it is possible to determine i) which dataelements to deliver to the Renderers to begin the presentation process;and ii) given that the presentation process has reached a certain point,which data elements to deliver to the Renderers next to continue thepresentation process; and (h) associated with each element is aRendition Period—the Rendition Period is the length of time therendering process should last for that element, where the RenditionPeriod of an element may be specified in many different ways, includingbut not limited to the following: (i) as a value explicitly stored aspart of the element, (ii) as a fixed value associated with that type ofdata element, and stored in a header field of the Presentation Data,(iii) as a fixed value associated with a Presentation System, (iv) adifference between the Data Time of the element and the Data Time of afollowing element that would be submitted to the same Renderer in thecourse of presentation (i.e., the element is rendered until there isanother element to be rendered by the same Renderer), (v) as a fixedproperty of the rendering process.

As defined herein, a Variable Rate Presentation (“VRP”) Module (alsoknown as a VRP Module) means a module that: (a) accepts control commandsand messages from the Presentation System; (b) maintains and, inresponse to commands from the Presentation System, updates the value ofa Current Presentation Rate parameter where values of this parameterhave the following interpretation: (i) a value of 1.0 means thatpresentation is to take place at the “normal” or default rate, (ii) avalue less than 1.0 but greater than zero means that presentation is totake place slower than “normal” (the factor by which presentation is tobe slowed down is equal to the inverse of the Current Presentation Ratevalue), (iii) a value greater than 1.0 means that presentation is totake place faster than “normal” (the factor by which presentation is tobe sped up is equal to the Current Presentation Rate value), and (iv) avalue less than zero is interpreted identically to the correspondingpositive value of the parameter, but the direction of presentation isreversed (i.e., the presentation “runs in reverse”); (c) processesTemporal Sequence Presentation Data; (d) modifies Temporal SequencePresentation Data in a manner consistent with the value of the CurrentPresentation Rate parameter and the Renderers to which the data will belater submitted, having the effect that the rate with which processingtakes places will track the value of the Current Presentation Rateparameter. The implementation of Variable Rate Presentation may alsoinvolve one or more Renderers. In this case the value of the CurrentPresentation Rate parameter is attached to the data elements, orotherwise communicated to the appropriate Renderers.

In a Presentation System fabricated in accordance with one embodiment ofthe present invention, the Presentation System would maintain twoseparate values of the Current Time parameter. The first of the valueswould be a Current Data Time value that indicates the largest Data Timevalue of any data element that has already been submitted for rendering,or should have already been submitted for rendering. The second of thevalues would be a Current Presentation Time value, which would be theCumulative Rendition Period of all data elements submitted sincepresentation began (i.e., the elapsed rendering time). In a PresentationSystem fabricated in accordance with another embodiment of the presentinvention, the Presentation System would maintain a single value of theCurrent Time parameter. Such an embodiment is typical of PresentationSystems that were not designed with the notion of variable rate playbackin mind. More specifically, some such systems were designed with animplicit assumption that the only possible presentation rate was 1.0.

Presentation Time and Data Time are identical properties in traditionalPresentation Systems, such as media players. However, when aPresentation System is enhanced with a Variable Rate Presentation(“VRP”) capability, these properties are no longer the same. We havediscovered that this presents a problem when a traditional media playeror other Presentation System is enhanced with a VRP capability, for tworeasons. First, if a Presentation System Control Module only acquires asingle Current Time value from a Timing Renderer, it is probablyassuming that that value can be interpreted as both Current Data Timeand Current Presentation Time. If these times are not equal, at leastone of those assumptions will be in error. Secondly, if a single CurrentTime value is distributed to multiple components, some of whichinterpret the value as Current Data Time, and some of which interpretthe value as Current Presentation Time, at least one of theseinterpretations will be in error. We have invented a way to convertPresentation Time to Data Time, and we have invented a method forreporting Presentation Time when only one time can be returned.

In accordance with one embodiment of the present invention, certaintime-related properties (that will later be used to calculate CurrentPresentation Time and Current Data Time) are associated with TemporalSequence Presentation Data elements by taking the following steps.

Step 1: from time to time as a user or some other controlling entitydecides to change a rate of presentation, the Presentation SystemControl Module sends a message to a Variable Rate Presentation Module(VRP Module), specifying an updated value for a Current PresentationRate parameter. When the VRP Module receives this message, it updatesthe value of its Current Presentation Rate parameter.

Step 2: in preparation for presentation, the Presentation Systemorganizes Temporal Sequence Presentation Data elements (for example,audio samples) into collections called buffers.

Step 3: buffers are presented in an ordered or semi-order fashion forPresentation Rate modification and rendering, typically according to theData Time of the first data element in a buffer.

Step 4: the number of unmodified samples in each buffer is determined,and the Unmodified Rendition Period of each element is obtained. Thevalue of the Current Presentation Rate parameter is held constant (i.e.,it is not allowed to change) while the VRP Module is processing abuffer. Also, the Rendition Period of all data elements in a buffer isconstrained to be equal. Note, however, if it were desired to varyeither the Current Presentation Rate or the Rendition Period within abuffer, that buffer could be broken down into multiple smaller buffersin which those properties were constant. In doing so, if necessary, eachbuffer could hold only a single data element.

Step 5: the Unmodified Cumulative Rendition Period for the buffer iscalculated and retained as a property of the buffer by multiplying theRendition Period of each data element in the buffer by the number ofunmodified data elements in the buffer.

Step 6: the Data Time of the buffer is defined to be the Data Timeassociated with the first unmodified data element in the current buffer.The Data Time is acquired or calculated, and retained as a property ofthe buffer. If it is not directly specified as a property of the firstdata element of the current buffer, it can be calculated by adding theData Time of the previous buffer to the Unmodified Cumulative RenditionPeriod of the previous buffer.

Step 7: the data elements in the current buffer are presentation ratemodified, so that the ratio of the Cumulative Rendition Period of thebuffer prior to presentation rate modification, divided by theCumulative Rendition Period of the buffer following modification, issubstantially equal to the Current Presentation Rate.

Step 8: the number of modified data elements in the modified buffer, andthe Modified Rendition Period of each data element in the buffer, isdetermined and retained as a property of the buffer.

Step 9: the Modified Cumulative Rendition Period for the buffer iscalculated and retained as a property of the buffer by multiplying theModified Rendition Period of each data element in the buffer by thenumber of modified data elements in the buffer.

Step 10: the Modified Presentation Time of the buffer is defined to bethe Presentation Time associated with the first modified data element inthe buffer. This time is calculated and retained as a property of thebuffer. It is calculated by adding to the Modified Presentation Time ofthe first modified data element of the previous buffer, the ModifiedCumulative Rendition Time of the previous buffer.

Step 11: calculate, and retain as a property of the current buffer, theCumulative Modified Data Element Count associated with the first dataelement in the current buffer by adding the Cumulative Modified DataElement Count associated with the first modified data element in theprevious buffer to the number of modified data elements in the previousbuffer.

Note that in this embodiment only the VRP Module needs to be informed ofthe current value of the Presentation Rate parameter. Renderer Modules,on the other hand, get all of their information about Presentation Ratefrom the buffer properties described above.

In accordance with one embodiment of the present invention, eachRenderer is assumed to comprise a Core component. This Core componentmay be hardware, software, or a combination of hardware and software.For example, the Core component of an audio Renderer may be a Sound Cardand its associated driver software. The Core component performs theessential rendering process for the particular Type of Temporal SequencePresentation Data that the Renderer processes.

In accordance with one embodiment of the present invention, the Corecomponent can be asked at any point in time to report the number of dataelements rendered since a distinguished event such as, for example, aninvocation of an Initialize or Clear command. Equivalently, the Corecomponent rendering hardware or software may be able to report thenumber of milliseconds of rendering that has occurred since adistinguished event.

Renderers, especially Timing Renderers, must decide how to respond whenother components of the Presentation System ask for the Current Timevalue without specifying whether Presentation Time or Data Time isdesired. In many cases it is possible to determine that the PresentationSystem really wants to know what the Current Data Time is. Therefore inaccordance with one embodiment of the present invention, a certain DataTime is returned when a request is made for the Current Time. For this,and other reasons, Renderers, especially Timing Renderers, must maintainan up-to-date and accurate value for both the Presentation Time and theData Time associated with the data element currently being rendered. Inaccordance with one embodiment of the present invention, it is the DataTime of the data element currently being rendered by the Core componentthat is returned as the Current Time when the Current Time is requestedby another module. Therefore, in accordance with one embodiment of thepresent invention, Current Presentation Time and Current Data Time arecalculated by taking the following steps.

Step 1: a modified buffer with its associated properties as describedabove is submitted to an appropriate Renderer.

Step 2: if the Renderer's Core component is capable of reporting thenumber of milliseconds of rendering that has occurred since adistinguished event, and the Modified Presentation Time of this modifiedbuffer is zero (or some other distinguished value), the Renderertriggers the distinguished event in its Core component.

Step 3: the Renderer submits the contents of the buffer to its Corecomponent.

Step 4: the Renderer also stores each modified buffer in some methodthat allows ready access to all of the buffer properties until it hasdetermined that all data elements in the buffer have been rendered.

Step 5: if the Core component is capable of reporting the number of dataelements rendered since a distinguished event occurred, the Renderercalculates the Current Data Time and the Current Presentation Time usingthe following algorithm.

Step 5a: it asks its Core component for the cumulative number of dataelements rendered.

Step 5b: it determines which buffer the next data element to be renderedwill have come from, by identifying the particular buffer that has forits Cumulative Modified Data Element Count the largest cumulative samplenumber less than or equal to the reported number of data elementsrendered—this buffer is referred to as the current rendering buffer.

Step 5c: the current Data Time is equal to the Data Time of the currentrendering buffer, plus an offset.

Step 5d: the offset is calculated by multiplying the unmodified bufferduration by the Completion Fraction.

Step 5e: the Completion Fraction is calculated by subtracting thecumulative sample number associated with the first sample in the currentrendering buffer from the Core component's reported number, and dividingthe result by the number of modified samples in the buffer.

Step 5f: the Current Presentation Time is equal to the ModifiedPresentation Time of the current rendering buffer, plus an offset.

Step 5g: the offset is calculated by multiplying the buffer's ModifiedCumulative Rendition Period by the Completion Fraction.

Step 6: if the Core is capable of reporting the number of millisecondsof rendering that has occurred since a distinguished event, the Renderercalculates the Current Data Time and the Current Presentation Time usingthe following algorithm.

Step 6a: it asks its Core component for the number of milliseconds ofrendering that has occurred.

Step 6b: it determines which buffer the next data element to be renderedwill have come from by identifying the particular audio buffer that hasfor its Modified Presentation Time the largest value less than or equalto the Core's reported value—this buffer is referred to as the currentrendering buffer.

Step 6c: the current Data Time is equal to the Data Time of the currentrendering buffer, plus an offset.

Step 6d: the offset is calculated by multiplying the unmodified bufferduration by the Completion Fraction.

Step 6e: the Completion Fraction is calculated by subtracting theModified Presentation Time of the current rendering buffer from the Corecomponent's reported value, and dividing the result by the CumulativeModified Rendering Period of the buffer.

Step 6f: the Current Presentation Time is equal to the ModifiedPresentation Time of the current rendering buffer, plus an offset.

Step 6g: the offset is calculated by subtracting the ModifiedPresentation Time of the current rendering buffer from the Corecomponent's reported value.

Step 6h: the current Data Time is reported to the player as thePresentation Time.

Step 7: whenever an object requests the Current Time, the Renderercomputes an updated value for the Presentation Time and Data Time, andreports either or both times as appropriate.

Let us define a temporal processing element as a sequence of TemporalSequence Presentation Data samples over which the Current PresentationRate does not change from one sample in the sequence to the next, andthe default Rendition Period does not change from one sample in thesequence to the next. Such an element may comprise one or more datasamples. Any sequence of Temporal Sequence Presentation Data samples canthus be considered to be organized into a sequence of temporalprocessing elements, each comprising one or more samples.

Referring to FIG. 3A, a flowchart is shown of a method 300 foridentifying the values of various presentation-related parametersaccording to one embodiment of the present invention. Note that thesteps illustrated in FIG. 3A need not be performed in the sequenceillustrated. Furthermore, steps may be added to and/or removed from themethod 300 illustrated in FIG. 3A to identify the values of desiredproperties.

The default period b_(d) of an element is the Rendition Period of suchan element when it is rendered at normal speed. Thus b_(d) is the ratioof dτ, the change in Data Time, (which may be identified in step 302)and de, the change in the index E of the current element being rendered(which may be identified in step 304), as shown in Equation 1 (step306).

b _(d) =dτ/de   Equation 1

The presentation rate r is equal to the ratio of dτ, the change in DataTime, and dt, the change in Presentation Time, (which may be identifiedin step 308), as shown in Equation 2 (step 310).

r=dτ/dt   Equation 2

The actual period b_(a) of an element is the Rendition Period of such anelement when it is rendered at the current Presentation Rate. Thus b_(a)is the ratio of dt, the change in Presentation Time, and de, the changein the index E of the current element being rendered, as shown inEquation 3 (step 312).

b _(a) =dt/de=b _(d) /r   Equation 3

At any instant in the rendition of Temporal Sequence Presentation Data,the relationship between b_(d), b_(a), r, the element index E, the DataTime T_(D), and the Presentation Time T_(P), can be expressed inintegral form. For a system in which the initial value of both Data Timeand Presentation Time are zero before playback begins, the current DataTime T_(D) at Presentation Time T_(P) is given by Equation 4:

T _(D)=∫₀ ^(T) ^(P) dτ=∫ ₀ ^(T) ^(P) rdt   Equation 4

Therefore, the current Data Time T_(D) at Presentation Time T_(P) may beidentified using the method 320 illustrated in FIG. 3B. The currentPresentation Time T_(P) is identified (step 322) and the current DataTime T_(D) is identified either by integrating over dτ (step 324) orover rdt (step 326).

The current Data Time T_(D) at element index E is given by Equation 5:

T _(D)=∫₀ ^(E) dτ=∫ ₀ ^(E) rdt=∫₀ ^(E) r·b _(a) de=∫ ₀ ^(E) b _(d) de  Equation 5

Therefore, the current Data Time T_(D) at element index E may beidentified using the method 330 illustrated in FIG. 3C. The currentelement index E is identified (step 332) and the current Data Time T_(D)is identified either by integrating over dτ (step 334), rdt (step 336),rb_(a)de (step 338), or b_(d)de (step 339).

The current Presentation Time T_(P) at Data Time T_(D) is given byEquation 6:

T _(P)=∫₀ ^(T) ^(D) dt=∫ ₀ ^(T) ^(D) r ⁻¹ dτ  Equation 6

Therefore, the current Presentation Time T_(P) at Data Time T_(D) may beidentified using the method 340 illustrated in FIG. 3D. The current DataTime T_(D) is identified (step 342) and the current Presentation TimeT_(P) is identified either by integrating over dt (step 344) or overr⁻¹dτ (step 346).

The current Presentation Time T_(P) at element index E is given byEquation 7:

P _(P)=∫₀ ^(E) dt=∫ ₀ ^(E) r ⁻¹ dpτ=∫ ₀ ^(E) (b _(d) /r)de=∫ ₀ ^(E) b_(a) de   Equation 7

Therefore, the current Presentation Time T_(P) at element index E may beidentified using the method 350 illustrated in FIG. 3E. The currentelement index E is identified (step 352) and the current PresentationTime T_(P) is identified either by integrating over dt (step 354), r⁻¹dτ(step 356), (b_(d)/r)de (step 358) , or b_(a)de (step 359).

Those having ordinary skill in the art will understand how to addarbitrary constants to either the Data Time or Presentation Timecalculations above to accommodate situations in which the initial valuesof those times are nonzero.

In one embodiment of a system in which Data Time and Presentation Timevalues may differ, each sample buffer is considered to be a temporalprocessing element. A Current Presentation Rate parameter r isassociated with each such element. A Modified Cumulative RenditionPeriod b_(a), and an Unmodified Cumulative Rendition Period b_(d), isassociated with each element.

Referring to FIG. 3F, a flowchart is shown of a method 360 that may beused in such an embodiment to identify the value of b_(d) for aparticular element and for identifying the Data Time T_(D) at thebeginning of the element. The Modified Cumulative Rendition Period b_(a)for the element is identified (step 362). The Current Presentation Rater associated with the element is identified (step 364). The value ofb_(d) for the element is identified as the product of b_(a) and r forthat element (step 366). The Data Time T_(D) at the beginning of theelement is identified as the sum of the values of b_(d) for all of thepreceding elements (step 368). Note that the values of b_(d) for thepreceding elements may be identified by iteratively applying steps362-366 to those elements.

Referring to FIG. 3G, a flowchart is shown of a method 370 that may beused in such an embodiment to identify the value of b_(a) for aparticular element and for identifying the Presentation Time T_(P) atthe beginning of the element. The Unmodified Cumulative Rendition Periodb_(d) for the element is identified (step 372). The Current PresentationRate r associated with the element is identified (step 374). The valueof b_(a) for the element is identified as the quotient of b_(d) and rfor that element (step 376). The Presentation Time T_(P) at thebeginning of the element is identified as the sum of the values of b_(a)for all of the preceding elements (step 378). Note that the values ofb_(a) for the preceding elements may be identified by iterativelyapplying steps 372-376 to those elements.

It is not necessary to store both b_(d) and b_(a) for an element. Eithervalue can be stored, and the other calculated from that stored value andthe value of r for the element. Alternatively, the value of both b_(d)and b_(a) for an element may be stored, and r calculated as b_(d)divided by b_(a).

In one embodiment of a system in which Data Time and Presentation Timevalues may differ, the sequence of temporal processing elements that arerendered are assigned positive index values, so that the nth elementrendered has index value n. A Current Presentation Rate parameter rn isassociated with the nth such element. A Modified Cumulative RenditionPeriod b_(a,n), and an Unmodified Cumulative Rendition Period b_(d,n),is associated with the nth element. The Presentation Time at thebeginning of the nth element is T_(P,n), and the Data Time at thebeginning of the nth element is T_(D,n). An Average Presentation Rateparameter R_(n) is associated with the nth element. This parameter isdefined as shown in Equation 8:

R _(n) =T _(D,n) /T _(P,n)   Equation 8

In such an embodiment, the Data Time parameter T_(D,n) does not have tobe explicitly maintained by the system, but instead may be calculated asneeded as shown, for example, in Equation 9:

T _(D,P) =T _(P,n) ·R _(n)   Equation 9

Furthermore, R_(n), the value of the Average Rate parameter of the nthelement, may be calculated without identifying the values of the DataTime parameter or the Unmodified Cumulative Rendition Period of any ofthe elements presented, as shown, for example, in Equation 10:

$\begin{matrix}{{R_{1} = 1}{R_{n} = {{{( \frac{T_{P,{n - 1}}}{T_{P,n}} )R_{n - 1}} + {( \frac{b_{a,{n - 1}}}{T_{P,n}} )r_{n - 1}\text{:}\mspace{14mu} {for}\mspace{14mu} n}} > 1}}} & {{Equation}\mspace{14mu} 10}\end{matrix}$

Alternatively, the Presentation Time parameter T_(P,n) does not have tobe explicitly maintained, but instead may be calculated as needed asshown in Equation 11:

T _(P,n) =T _(D,n) ·R _(n) ⁻¹   Equation 11

In Equation 11, R_(n) ⁻¹ is the inverse of the Average Rate parameter ofthe nth element. R_(n) ⁻¹ may be calculated without identifying thevalues of the Presentation Time parameter or the Modified CumulativeRendition Period for any of the elements presented, as shown, forexample, in Equation 12:

$\begin{matrix}{{R_{1}^{- 1} = 1}{R_{n}^{- 1} = {{{( \frac{T_{D,{n - 1}}}{T_{D,n}} )R_{n - 1}^{- 1}} + {( \frac{b_{d,{n - 1}}}{T_{D,n}} )r_{n - 1}^{- 1}\text{:}\mspace{14mu} {for}\mspace{14mu} n}} > 1}}} & {{Equation}\mspace{14mu} 12}\end{matrix}$

When a component of a traditional player system that has been enhancedto support variable speed playback requests the value of the CurrentTime from a Timing Renderer (or other component capable of providingtemporal information), several issues may be considered in order toreturn an appropriate value for the Current Time. The appropriate valuefor the Timing Renderer to return may differ from one request toanother.

First, the Timing Renderer may determine whether to return a time valuebased on the current Data Time or a time value based on the currentPresentation Time. In a traditional player, these two times are thesame, but in a player capable of variable speed playback, or simplycapable of differentiating between Data Time and Presentation Time,those two values are in general different. In some circumstances, it maybe desirable to respond to a request for the Current Time with a timevalue that is based on the current Presentation Time. In othercircumstances, however, it may be desirable to respond with a time valuethat is based on the current Data Time. The performance of the system isenhanced, therefore, in various embodiments of the present invention byenabling the Timing Renderer to return a time value that is derived froman appropriate one of the current Data Time and current Presentationtime, based on the circumstances.

Moreover, it is commonly the case that the requesting component willeither add an offset to the time returned, thus in effect calculating anearlier or later time, or subtract the time returned from another time,thus in effect calculating a temporal differential. In many cases thefunction performed by such an addition or subtraction is such that thecalculated time or temporal differential would benefit from beingadjusted by taking into account the current presentation rate, or somevalue determined by the course of previous or future variations inpresentation rate. But because the requesting component is not awarethat playback speeds are variable, it does not make such an adjustment.

In summary, in various embodiments of the present invention in which atraditional player system is enhanced to support variable speedplayback, the performance of the system may be improved by ensuring thatthe Timing Renderer determines whether to base the time value returnedon the Current Presentation Time or the Current Data Time when acomponent asks for the Current Time. Furthermore, in various embodimentsof the present invention, the time returned is adjusted to take intoaccount the current presentation rate, or a sequence of variations inpresentation rates, to further improve system performance.

In general, an advantageous strategy for the Timing Renderer to employin responding to an ambiguous request for the Current Time is toidentify which of a number of known “time value request cases” thecurrent request is an instance and, based on this identification, toselect an algorithm to use to calculate an appropriate time value toreturn. In general an appropriate time value algorithm may make use of abase time and a set of algorithm-specific parameters whose values theTiming Renderer may identify on a case-by-case base. This strategy isshown in a flow chart in FIG. 4.

The Timing Renderer receives a request for a Current Time value (step402). The Timing Renderer identifies relevant characteristics of thetiming request (referred to herein as the “distinguishing signature” ofthe request) (step 404). The Timing Render identifies the known timevalue request case of which the current request is an instance (step406).

The Timing Renderer selects, based on the identified distinguishingsignature and request case, an algorithm to use to calculate anappropriate time value to return (step 408). The Timing Rendereridentifies an appropriate time value base and parameters for use withthe identified algorithm based on the distinguishing signature andrequest case (step 410). The Timing Renderer uses the identifiedalgorithm, base, and parameters to calculate a time value (step 412).The Timing Renderer returns the time value identified in step 412 to therequesting component (step 414).

In one set of cases, the appropriate base time will always be either theCurrent Data Time or the Current Presentation Time, and there will betwo relevant parameters: an incorrect offset (or differential) that theTiming Renderer anticipates the requesting component will add (orcompute), and a correct offset (or differential) that the TimingRenderer anticipates that the requesting component intends to add (orcompute). Frequently, the requesting component will employ an incorrectoffset, or incorrectly calculate a differential, when the Current Timeon which it is basing its calculation is the Data Time (PresentationTime), and the offset it intends to add, or differential it intends tocalculate, is intrinsically the Presentation Time (Data Time).

In these cases, an advantageous method 500 for the Timing Renderer toemploy in responding to the request is illustrated by the flowchartshown in FIG. 5. In general, in the method 500 the Timing Rendererdetermines, from the distinguishing signature of the request, which oneof a number of standard time value request cases the current request isan instance of. Then, based on the identification of the time valuerequest case, the Timing Renderer:

-   -   (A) selects either the current Data Time or the current        Presentation Time as the correct base time;    -   (B) determines whether it is anticipated that the requesting        component will add an incorrect offset to the time value        returned, or calculate an erroneous differential from the time        returned; and, if so,    -   (C) adds to the base time value the correct offset or        differential that the Timing Renderer anticipates, based on the        request signature, that the requesting component intends to add        or compute; and    -   (D) subtracts from the base value the incorrect offset or        differential that the Timing Renderer anticipates, based on the        request signature, that the requesting component will actually        add or compute.

More specifically, the method 500 determines whether the distinguishingrequest signature signals a custom time value request case (step 502).In one embodiment of the present invention, cases not falling within oneof the set of cases described below with respect to steps 514, 516, 520,522, 528, 530, 534, and 536 are regarded as custom cases for whichcustom time value calculation algorithms and parameters to perform thosealgorithms are identified on a case-by-case basis (step 504).

If the distinguishing request signature does not signal a custom timevalue request case, the method 500 determines whether the signatureidentifies a Presentation Time-based timing request (step 506). If thedistinguishing request signature does identify a Presentation Time-basedtiming request, the method 500 uses Presentation Time as a base time.The method 500 determines whether the distinguishing request signatureidentifies a comparison-type timing request (step 508).

If the distinguishing request signature identifies a comparison-typetiming request, the method 500 determines whether the intent of thecalculation that the requesting component is expected to perform is todetermine a Presentation Time differential Δ_(P) (step 510). If therequested component intends to calculate a Presentation Timedifferential, then the method 500 identifies the current case as the“PCP” case (step 514). The meaning of this case and other cases will bedescribed in more detail below. If the intent of the calculation thatthe requesting component is expected to perform is to determine a DataTime differential Δ_(P), then the method 500 identifies the current caseas the “PCD” case (step 516).

If the distinguishing request signature does not identify acomparison-type timing request, the method 500 determines whether therequesting component is expected to calculate a Presentation Time offsetO_(P) (step 518). If the requesting component is expected to calculate aPresentation Time offset, then the method 500 identifies the currentcase as the “POP” case (step 520). If the requesting component is notexpected to calculate a Presentation Time offset, then the method 500identifies the current case as the “POD” case (step 522).

Returning to step 506, if the distinguishing request signature does notidentify a Presentation Time-based timing request, the method 500 usesData Time as a base time. The method 500 determines whether thedistinguishing request signature identifies a comparison-type timingrequest (step 524). If the distinguishing request signature identifies acomparison-type timing request, the method 500 determines whether theintent of the calculation that the requesting component is expected toperform is to determine a Data Time differential Δ_(D) (step 526). Ifthe requesting component intends to calculate a Data Time differential,then the method 500 identifies the current case as the “DCD” case (step528). If the intent of the calculation that the requesting component isexpected to perform is to determine a Presentation Time differentialΔ_(P), the method 500 identifies the current case as the “DCP” case(step 530).

Returning to step 524, if the distinguishing request signature does notidentify a comparison-type timing request, the method 500 determineswhether the requesting component is expected to calculate a Data Timeoffset O_(D) (step 532). If the requesting component intends tocalculate a Data Time offset, then the method 500 identifies the currentcase as the “DOD” case (step 534). If the requesting component intendsto calculate a Presentation Time offset, then the method 500 identifiesthe current case as the “DOP” case (step 536).

For example, suppose that the requesting component is attempting todetermine what the value of the Presentation Time was when the Data Timewas 2.5 seconds earlier than the current moment. Further suppose that itdetermines this value by requesting the current time, and subtracting2.5 seconds from the value returned. Consider the case in which thePresentation Rate r is now 2.0 (signifying a playback rate that is twicethe default or normal playback rate), and has had that value for thepast minute. Then the optimal value for the Timing Renderer to return isa value that is based on the Presentation Time. Further, the optimalvalue for the Timing Renderer to return (in seconds) is given byEquation 13:

T ₀ =T _(P)+(−2.5/r)−(−2.5)   Equation 13

If the Timing Renderer returns T₀, and the requesting component adds anoffset of (−2.5 seconds), the final number calculated will be given byEquation 14, which results in the desired value.

T _(F) =T ₀−2.5=T _(P)+(−2.5/r)−(−2.5)+(−2.5)=T _(P)+(−2.5)/r   Equation14

In all of the cases mentioned above with respect to steps 514, 516, 520,522, 528, 530, 534, and 536, three values are identified: a base time, a“right” offset, and a “wrong” offset. The “right” offset is an offsetthat, when added to the base time, returns an appropriate time value toreturn in response to the time request received by the Timing Renderer.The “wrong” offset is an offset that the Timing Renderer expects therequesting component to add to the time value returned by the TimingRenderer. To compensate for this expected “wrong” offset, the TimingRenderer returns a time value that is equal to the base time plus theright offset minus the wrong offset (step 538). If the requestingcomponent adds the wrong offset to the returned value, as expected, theresulting value will be equal to the base time plus the right offset,which is the desired time value in response to the request.

For example, in the “DOP” case (step 536) the value desired by therequesting component when the Current Time is requested is the currentData Time, and the requesting component will intend to add a “right”Presentation Time offset O_(P) to the value returned, but, it isanticipated, will add a “wrong” Data Time offset O_(D). In this case, inone embodiment the Timing Renderer returns the current Data Time, plusthe right offset O_(P), minus the wrong offset O_(D).

In the “DOD” case (step 534), the value desired by the requestingcomponent when the Current Time is requested is the current Data Time,and the requesting component will intend to add a “right” Data Timeoffset OD to the value returned, and, it is anticipated, will actuallyadd the proper offset. In this case, the in one embodiment the TimingRenderer returns the current Data Time, without any adjustment.

In the “DCP” case (step 530), the value desired by the requestingcomponent when the Current Time is requested is the current Data Time,and the requesting component will compare the time returned with anotherData Time value that differs by an amount Δ_(D), and, it is anticipated,will interpret the difference between those two values as PresentationTime differential Δ_(P). In this case, in one embodiment the TimingRenderer returns the current Data Time, plus the right differentialΔ_(P), minus the wrong differential Δ_(D).

In the “DCD” case (step 528), the value desired by the requestingcomponent when the Current Time is requested is the current Data Time,and the requesting component will compare the time returned with anotherData Time value that differs by an amount Δ_(D), and, it is anticipated,will correctly interpret the difference between those two values as aData Time differential. In this case, in one embodiment the TimingRenderer returns the current Data Time, without any adjustment.

In the “POP” case (step 520), the value desired by the requestingcomponent when the Current Time is requested is the current PresentationTime, and the requesting component will intend to add a “right”Presentation Time offset O_(P) to the value returned, and, it isanticipated, will actually add the proper offset. In this case, in oneembodiment the Timing Renderer returns the current Presentation Time,without any adjustment.

In the “POD” case (step 522), the value desired by the requestingcomponent when the Current Time is requested is the current PresentationTime, and the requesting component will intend to add a “right” DataTime offset OD to the value returned, but, it is anticipated, will add a“wrong” Presentation Time offset O_(P). In this case, in one embodimentthe Timing Renderer returns the current Presentation Time, plus theright offset O_(D), minus the wrong offset O_(P).

In the “PCD” case (step 516), the value desired by the requestingcomponent when the Current Time is requested is the current PresentationTime, and the requesting component will compare the time returned withanother Presentation Time value that differs by an amount Δ_(P), and, itis anticipated, will interpret the difference between those two valuesas Data Time differential Δ_(D). In this case, in one embodiment theTiming Renderer returns the current Presentation Time, plus the rightdifferential Δ_(D), minus the wrong differential Δ_(P).

In the “PCP” case (step 514), the value desired by the requestingcomponent when the Current Time is requested is the current PresentationTime, and the requesting component will compare the time returned withanother Presentation Time value that differs by an amount Δ_(P), and, itis anticipated, will correctly interpret the difference between thosetwo values as a Presentation Time differential. In this case, in oneembodiment the Timing Renderer returns the current Presentation Time,without any adjustment.

In all cases for which the Timing Renderer calculates an offset ordifferential, any of the techniques discussed above for converting DataTimes to Presentation Times, or Presentation Times to Data Times, may beemployed as appropriate. In some cases the Timing Renderer may be ableto predict that a requesting component will later attempt to identifythe Data Time or Presentation Time associated with a moment that theTiming Renderer can identify. The Timing Renderer may choose to storeeither or both of the identified moment's Data Time and PresentationTime, and will be able to employ those stored times directly incomputing a later “right” offset and “wrong” offset.

In some cases the values of O_(P), O_(D), Δ_(P), or Δ_(D) that a TimingRenderer might identify for a timing value request may not be equal tothe actual values that the requesting component might employ followingthe request. It is frequently the case in engineered systems that exactvalues cannot be determined precisely, and components must use estimatesof values. The values of O_(P), O_(D), Δ_(P), or Δ_(D) may be estimatedusing any appropriate techniques, including those that are well-known tothose having ordinary skill in the art.

As indicated above with respect to FIGS. 4-5, whenever a Timing Rendererreceives a request for the Current Time, the Timing Renderer maydetermine which of the time value request cases this is an instance of,so as to be able to return an appropriate time value. The TimingRenderer may, for example, determine which case this request is aninstance of by identifying a distinguishing signature associated withthe current request, and using the distinguishing signature to identifya corresponding time value request case. Once the Timing Renderer hasidentified the corresponding time value request case, it may employ thatidentification to determine whether to base the return value on DataTime or Presentation Time. The Timing Renderer may also use the requestcase identification to determine how to adjust the selected base time.For instance, the Timing Renderer may base its estimate of O_(P), O_(D),Δ_(P), or Δ_(D) on the request signature and/or the request case.

The distinguishing signature may be identified from one or moreproperties of the request itself. In addition, properties related to thestate of the player or other properties of the rendering system that areaccessible to the Timing Renderer may contribute to the signature. Thedistinguishing signature may be an arbitrary combination of manyproperties, including but not limited to one or more of the following:

-   -   The return address of the subroutine call from the requesting        component to the Timing Renderer (if the request is made via a        subroutine call), or more generally a pattern present in the        stack frame of the calling process or thread (for example the        return address of the third subroutine back in the stack frame);    -   The specific subclass of the Timing Renderer through which the        requesting component makes its request (in the case of an        object-oriented request);    -   The identity of the message by which the timing request is made,        if a message-passing protocol is used to make the timing        request;    -   The identity of the execution thread or process within which the        request is being made;    -   The phase, stage, or state of the player thread, process, or        application when the timing request is made;    -   A distinguished or relevant pattern in the player memory;    -   A preference value set by the user or an external monitor        program or tuning algorithm;    -   The number, timing, or specific pattern of prior calls to the        Timing Renderer;    -   The name, source, encoding format, or other characteristic of        the data currently being rendered;    -   The name of the program being executed, the user's name, or the        identity or make of the computer on which the program is being        executed;    -   A value accessible over the Internet, located in the memory of        the computer running the player program, present on a hard disk        or other data storage device, or otherwise directly or        indirectly accessible to the Timing Renderer;    -   The state of a button, lever, slider, switch, or other input        device directly or indirectly accessible to the Timing Renderer;    -   The hardware characteristics of the platform on which the        program is running, including but not limited to the size of        memory, the identity or characteristics of the processor, or the        size of external storage; and    -   The source of the data being rendered, or the nature of the        channel between the data source and the player. For instance, in        the case of media streaming over the Internet, the URL of the        source material, or the capacity of the Internet connection, may        constitute a factor.

Note that the term “property of the request” refers herein to anyproperty from which the distinguishing signature of the request isderived. As the examples above indicate, such properties need not berepresented in the request itself. For example, the state of any part ofthe rendering system may be treated as a “property of the request” forpurposes of identifying the distinguishing signature of the request. Thedetermination of the properties which comprise the distinguishingsignature from which the time value request case is identified, and thedetermination of the time value parameters from which the final timevalue is calculated, may be accomplished in multiple ways, including butnot limited to the following:

-   -   Examining the source code of the player program;    -   Analyzing the operating characteristics of the player program        when playing media at various speeds;    -   Analyzing debugging output, error messages, and performance logs        generated by the player program;    -   Providing a user interface by which users of the program may        identify the properties which distinguish one request case from        another, and by which users may specify the values of the time        value parameters to be used with each request case;    -   Implementing an automatic optimization technique for traversing        the search space of possible distinguishing properties and        parameter values, identifying a metric for determining a        goodness-of-fit value for each point in the search space, and        running an optimization algorithm for identifying the best        location in the search space.

Using any of the techniques above, or others that will occur to personsof ordinary skill in the art, a specification may be determined ofrelevant properties that together form a distinguishing requestsignature. The request signature then provides the mapping to acorresponding timing request case, and constitutes the means by which acorresponding set of time value parameters may be identified. The timevalue request case identity and the time value parameters togetherdetermine and fully specify the corresponding time value for the TimingRenderer to calculate and to return to the requesting component.

It is to be understood that although the invention has been describedabove in terms of particular embodiments, the foregoing embodiments areprovided as illustrative only, and do not limit or define the scope ofthe invention. Various other embodiments, including but not limited tothe following, are also within the scope of the claims. For example,elements and components described herein may be further divided intoadditional components or joined together to form fewer components forperforming the same functions.

The techniques described above may be implemented, for example, inhardware, software, firmware, or any combination thereof. The techniquesdescribed above may be implemented in one or more computer programsexecuting on a programmable computer including a processor, a storagemedium readable by the processor (including, for example, volatile andnon-volatile memory and/or storage elements), at least one input device,and at least one output device. Program code may be applied to inputentered using the input device to perform the functions described and togenerate output. The output may be provided to one or more outputdevices.

Each computer program within the scope of the claims below may beimplemented in any programming language, such as assembly language,machine language, a high-level procedural programming language, or anobject-oriented programming language. The programming language may, forexample, be a compiled or interpreted programming language.

Each such computer program may be implemented in a computer programproduct tangibly embodied in a machine-readable storage device forexecution by a computer processor. Method steps of the invention may beperformed by a computer processor executing a program tangibly embodiedon a computer-readable medium to perform functions of the invention byoperating on input and generating output. Suitable processors include,by way of example, both general and special purpose microprocessors.Generally, the processor receives instructions and data from a read-onlymemory and/or a random access memory. Storage devices suitable fortangibly embodying computer program instructions include, for example,all forms of non-volatile memory, such as semiconductor memory devices,including EPROM, EEPROM, and flash memory devices; magnetic disks suchas internal hard disks and removable disks; magneto-optical disks; andCD-ROMs. Any of the foregoing may be supplemented by, or incorporatedin, specially-designed ASICs (application-specific integrated circuits)or FPGAs (Field-Programmable Gate Arrays). A computer can generally alsoreceive programs and data from a storage medium such as an internal disk(not shown) or a removable disk. These elements will also be found in aconventional desktop or workstation computer as well as other computerssuitable for executing computer programs implementing the methodsdescribed herein, which may be used in conjunction with any digitalprint engine or marking engine, display monitor, or other raster outputdevice capable of producing color or gray scale pixels on paper, film,display screen, or other output medium.

What is claimed is:
 1. A method, performed by at least one machine, forrendering temporal sequence presentation data in a machine-implementedrendering system, the temporal sequence presentation data being tangiblystored in a first computer-readable medium and comprising an unmodifiedplurality of n data elements corresponding to a portion of the temporalsequence presentation data, wherein each of the data elements in theunmodified plurality share a fixed specified presentation duration, themethod comprising steps of: (A) generating a modified plurality of dataelements corresponding to the portion of the temporal sequencepresentation data; (B) maintaining a value of a presentation timeparameter tangibly stored in a second computer-readable medium andrepresenting an amount of time required to render the modified pluralityof data elements; (C) providing the value of the presentation timeparameter to a first component of the rendering system; and (D)providing a value of a data time parameter to a second component of therendering system, wherein the value of the data time parameterrepresents n multiplied by the fixed specified presentation duration;wherein the value of the presentation time parameter is not equal to thevalue of the data time parameter.